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Preface  &  Acknowledgements 


Welcome  to  our  Tenth  Annual  Acquisition  Research  Symposium!  We  regret  that  this 
year  it  will  be  a  “paper  only”  event.  The  double  whammy  of  sequestration  and  a  continuing 
resolution,  with  the  attendant  restrictions  on  travel  and  conferences,  created  too  much 
uncertainty  to  properly  stage  the  event.  We  will  miss  the  dialogue  with  our  acquisition 
colleagues  and  the  opportunity  for  all  our  researchers  to  present  their  work.  However,  we 
intend  to  simulate  the  symposium  as  best  we  can,  and  these  Proceedings  present  an 
opportunity  for  the  papers  to  be  published  just  as  if  they  had  been  delivered.  In  any  case,  we 
will  have  a  rich  store  of  papers  to  draw  from  for  next  year’s  event  scheduled  for  May  14-15, 
2014! 


Despite  these  temporary  setbacks,  our  Acquisition  Research  Program  (ARP)  here  at 
the  Naval  Postgraduate  School  (NPS)  continues  at  a  normal  pace.  Since  the  ARP’s 
founding  in  2003,  over  1 ,200  original  research  reports  have  been  added  to  the  acquisition 
body  of  knowledge.  We  continue  to  add  to  that  library,  located  online  at 
www.acquisitionresearch.net,  at  a  rate  of  roughly  140  reports  per  year.  This  activity  has 
engaged  researchers  at  over  70  universities  and  other  institutions,  greatly  enhancing  the 
diversity  of  thought  brought  to  bear  on  the  business  activities  of  the  DoD. 

We  generate  this  level  of  activity  in  three  ways.  First,  we  solicit  research  topics  from 
academia  and  other  institutions  through  an  annual  Broad  Agency  Announcement, 
sponsored  by  the  USD(AT&L).  Second,  we  issue  an  annual  internal  call  for  proposals  to 
seek  NPS  faculty  research  supporting  the  interests  of  our  program  sponsors.  Finally,  we 
serve  as  a  “broker”  to  market  specific  research  topics  identified  by  our  sponsors  to  NPS 
graduate  students.  This  three-pronged  approach  provides  for  a  rich  and  broad  diversity  of 
scholarly  rigor  mixed  with  a  good  blend  of  practitioner  experience  in  the  field  of  acquisition. 
We  are  grateful  to  those  of  you  who  have  contributed  to  our  research  program  in  the  past 
and  encourage  your  future  participation. 

Unfortunately,  what  will  be  missing  this  year  is  the  active  participation  and 
networking  that  has  been  the  hallmark  of  previous  symposia.  By  purposely  limiting 
attendance  to  350  people,  we  encourage  just  that.  This  forum  remains  unique  in  its  effort  to 
bring  scholars  and  practitioners  together  around  acquisition  research  that  is  both  relevant  in 
application  and  rigorous  in  method.  It  provides  the  opportunity  to  interact  with  many  top  DoD 
acquisition  officials  and  acquisition  researchers.  We  encourage  dialogue  both  in  the  formal 
panel  sessions  and  in  the  many  opportunities  we  make  available  at  meals,  breaks,  and  the 
day-ending  socials.  Many  of  our  researchers  use  these  occasions  to  establish  new  teaming 
arrangements  for  future  research  work.  Despite  the  fact  that  we  will  not  be  gathered 
together  to  reap  the  above-listed  benefits,  the  ARP  will  endeavor  to  stimulate  this  dialogue 
through  various  means  throughout  the  year  as  we  interact  with  our  researchers  and  DoD 
officials. 

Affordability  remains  a  major  focus  in  the  DoD  acquisition  world  and  will  no  doubt  get 
even  more  attention  as  the  sequestration  outcomes  unfold.  It  is  a  central  tenet  of  the  DoD’s 
Better  Buying  Power  initiatives,  which  continue  to  evolve  as  the  DoD  finds  which  of  them 
work  and  which  do  not.  This  suggests  that  research  with  a  focus  on  affordability  will  be  of 
great  interest  to  the  DoD  leadership  in  the  year  to  come.  Whether  you’re  a  practitioner  or 
scholar,  we  invite  you  to  participate  in  that  research. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  ARP: 
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Abstract 

In  the  face  of  both  declining  budgets  and  growing  interoperability  requirements,  the  military 
increasingly  wants  to  consolidate  multiple  needs  into  single  systems  to  be  developed  jointly. 
Unfortunately,  the  track  record  for  joint  system  acquisition  programs  is  mixed,  and  programs 
often  follow  a  familiar  downward  spiral: 

The  stakeholder  programs  that  depend  on  a  joint  system  may  be  skeptical,  fearing 
the  needed  capability  will  neither  meet  their  needs,  nor  be  delivered  as  promised. 
Stakeholders  pressure  the  Joint  Program  Office  (JPO)  to  accommodate  individual 
requirements,  and  the  JPO  may  reluctantly  agree,  driving  up  cost,  schedule, 
complexity,  and  risk — thus  realizing  the  stakeholders’  worst  fears.  These 
performance  issues  encourage  stakeholders  to  leave  the  joint  program,  potentially 
rendering  it  both  operationally  unattractive  and  financially  infeasible. 

This  exemplifies  a  classic  social  dilemma  called  the  “Tragedy  of  the  Commons.”  Much  work 
has  been  done  on  mitigating  social  dilemmas,  but  a  solution’s  success  depends  on  its 
context.  This  paper  describes  the  modeling  of  organizational  decision-making  in  a  joint 
acquisition  program  using  system  dynamics.  This  permits  future  work  to  analyze  the 
effectiveness  of  different  social  dilemma  mitigations  within  the  context  of  joint  programs  by 
using  system  dynamics. 


Introduction 

The  failure  of  acquisition  programs  to  deliver  high-quality  systems  within  cost  and 
schedule  constraints  (GAO,  2005) — especially  those  developing  software-reliant  systems — 
is  all  too  common  in  modern  government  acquisition.  These  recurring  failures  have  a  direct 
adverse  impact  on  the  ability  of  the  Department  of  Defense  (DoD)  to  be  able  to  support  the 
warfighter  with  the  systems  they  need.  Delayed  systems  withhold  needed  capability,  and 
wasted  resources  drain  budgets  that  could  be  used  to  develop  other  systems. 

The  Software  Engineering  Institute  (SEI)  has  a  unique  insight  into  these  failures  from 
regularly  conducting  Independent  Technical  Assessments  (ITAs)  on  specific  programs  to 
determine  why  they  are  experiencing  difficulties.  These  investigations  have  provided 
visibility  into  the  processes  and  forces  at  work  within  these  programs  and  have  produced  an 
understanding  of  the  most  common  ways  that  programs  come  to  face  serious  challenges. 
Acquisition  programs  do  not  fail  solely  for  technical  reasons.  Organizational,  management, 
and  cultural  issues  are  an  additional  set  of  significant  reasons  why  acquisition  programs 
may  substantially  exceed  budget,  overrun  schedule,  deliver  inadequate  quality,  and 
ultimately  even  fail  (Frangos,  1998;  Madachy,  2008). 

This  paper  describes  research  that  is  being  conducted  to  better  understand  the  joint 
acquisition  program  dilemma  and  to  investigate  approaches  to  mitigate  associated 
problems.  The  general  approach  is  to  use  a  causal  loop  diagram  (CLD)  as  a  means  to 
capture  a  current  understanding  of  the  problem  based  on  past  experience  in  both  consulting 
on  joint  programs  and  in  conducting  ITAs.  The  CLD  embodies  an  evolving  theory  of  the  joint 
acquisition  dilemma  that  is  updated  and  refined  through  a  series  of  workshops  held  with  joint 
program  domain  experts  and  decision-makers.  The  evolving  theory  is  further  explored  by 
developing  the  CLD  into  a  fully  executable  system  dynamics  model.  Data  collected  during 
workshops  help  to  guide,  correct,  and  validate  important  aspects  of  the  model.  When  the 
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model  adequately  captures  the  joint  program  dilemma,  it  can  be  used  to  investigate 
mitigations  to  the  problem  through  additional  modeling  of  different  mitigation  approaches. 
Ultimately,  the  most  promising  mitigations  can  be  evaluated  in  the  workshop  context  and 
potentially  in  pilot  tests  during  the  execution  of  actual  joint  acquisitions. 

The  subsequent  portions  of  this  paper  describe  the  progress  that  has  been  made  in 
conducting  this  research.  The  section  Social  Dilemmas  and  Joint  Programs  describes  the 
typical  flow  of  joint  acquisition  program  events.  The  section  System  Dynamics  Background 
provides  an  introduction  to  the  system  dynamics  modeling  approach.  The  section  Workshop 
with  Domain  Experts  describes  the  workshops  that  have  been  held  thus  far,  and  the  primary 
insights  gained.  The  section  The  Joint  Program  Simulation  Model  describes  the  current 
state  of  a  system  dynamics  simulation  model  refined  based  on  feedback  provided  during 
these  workshops.  Key  behaviors  exhibited  by  the  model  support  the  hypothesis  that  joint 
programs  suffer  from  the  “Tragedy  of  the  Commons”  social  dilemma  and  that  joint  program 
participants  may  get  caught  in  a  trap  that  can  lead  to  the  demise  of  the  program.  The 
section  Mitigations  for  the  Joint  Program  Dilemma  describes  the  space  of  potential 
mitigations  and  solutions  to  the  problems  illustrated.  Finally,  the  paper  concludes  with  a 
discussion  of  the  implications  of  this  work  and  some  future  opportunities. 

Joint  Programs 

The  category  of  programs  known  as  “joint”  programs  constitute  a  special  case  within 
DoD  acquisition.  Such  programs  intend  to  provide  a  system,  subsystem,  or  capability  that 
will  fulfill  needs  of,  and  be  funded  or  managed  by,  more  than  one  DoD  service  or 
component.  Joint  programs  are  appealing  because  they  offer  at  least  two  significant 
potential  benefits:  (1)  reducing  costs  by  developing  one  system  as  opposed  to  several 
differing  ones  and  (2)  improving  interoperability  by  providing  a  single  system  or  capability 
that  can  be  used  for  multiple  purposes  in  multiple  contexts.  Joint  programs  are  recognized 
as  being  difficult  to  manage  because  they  have  multiple  stakeholder  programs  intending  to 
use  the  joint  capability  (often  with  differing  needs),  they  may  be  larger  in  size  than  other 
programs,  they  may  be  more  complex  organizationally,  and  they  may  be  geographically 
dispersed — all  causing  increased  levels  of  coordination,  communication,  and  negotiation 
overhead.  At  the  same  time,  joint  programs  are  becoming  increasingly  important  to  the 
military  as  the  need  for  interoperability  grows  and  as  there  is  greater  pressure  on  the  overall 
defense  budget  to  reduce  costs. 

Although  the  focus  of  most  acquisition  programs  is  on  the  complex  system  being 
developed,  it  may  be  overlooked  that  acquisition  programs  themselves,  especially  joint 
programs,  are  complex,  dynamic  systems — and  as  such  can  display  unpredictable  and  even 
seemingly  chaotic  behavior.  This  results  from  the  presence  of  feedback  between  the 
autonomous  actors  populating  different  groups  within  the  acquisition  organization.  Feedback 
in  the  system  produces  non-linear  behavior,  where  changes  in  the  system’s  outputs  may  no 
longer  be  proportional  to  changes  to  the  inputs.  The  complexity  of  this  feedback,  inherent  in 
any  system  involving  interacting  human  beings,  coupled  with  time  delays  between  inputs 
and  outputs  that  obscure  the  relationships  between  cause  and  effect,  can  produce 
unexpected  behavior  in  even  simple  systems.  Such  systems  must  be  analyzed  as  a  whole 
in  order  to  understand  their  behavior,  because  the  problematic  behaviors  often  emerge 
directly  as  a  result  of  these  interactions — and  vanish  when  the  system  is  decomposed  into 
its  component  pieces  for  study. 

Misaligned  Incentives  in  Acquisition 

It  has  been  concluded  in  studies  (Kadish,  2006;  Pennock,  2008)  that  the  incentives 
at  work  in  acquisition  policy  and  governance  are  often  misaligned.  These  misalignments  can 
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cause  a  disconnect  between  the  desired  outcome  and  the  most  promising  ways  of  achieving 
that  outcome.  The  result  of  misaligned  incentives  can  be  shortsighted  acquisition  decision¬ 
making,  potentially  putting  short-term  interests  ahead  of  longer  term  interests,  or  individual 
and  program  interests  ahead  of  PEO  and  service  interests,  thus  turning  planned  cooperation 
into  opposition. 

Many  of  the  misaligned  incentives  seen  in  acquisition  belong  to  a  category  of 
problems  known  as  social  dilemmas.  Social  dilemmas  are  ubiquitous  across  human 
organizations.  They  describe  situations  in  which  the  incentives  align  to  promote  a  solution 
by  the  actors  involved  that  may  be  locally  optimal  but  will  be  suboptimal  at  a  more  global 
level. 


One  common  type  of  social  dilemma  is  called  a  “social  trap”  (Cross  &  Guyer,  1980; 
Kollock,  1998).  In  a  group  context,  a  social  trap  means  that  an  individual  desires  a  benefit  to 
himself  that  will  cost  everyone  else — but  if  all  in  the  group  succumb  to  the  same  temptation, 
then  everyone  is  worse  off.  A  social  trap  is  often  referred  to  colloquially  as  a  “Tragedy  of  the 
Commons”2  (Hardin,  1968).  What  is  noteworthy  about  this  dilemma  is  that  there  is  no  intent 
to  destroy  the  common  resource — it’s  the  combined  actions  of  all  acting  in  their  own  self- 
interest  that  lead  to  the  tragic  result. 

Social  dilemmas  come  in  many  different  forms,  with  many  different  properties,  which 
helps  to  make  them  both  difficult  to  recognize  and  difficult  to  fix.  The  next  section  outlines 
social  dilemmas  in  the  context  of  a  joint  program. 

Social  Dilemmas  and  Joint  Programs 

Joint  programs  are  noted  for  the  unique  challenges  that  they  face  organizationally 
(Lindsay,  2006),  due  in  part  to  the  tension  between  the  individual  programs  and  services 
needing  to  look  out  for  their  own  interests  and  the  Goldwater-Nichols  Department  of 
Defense  Reorganization  Act  of  1986  that  stresses  the  importance  of  all  service  branches 
working  together  both  effectively  and  efficiently.  Because  of  this  seeming  paradox,  there  is  a 
fundamental  social  dilemma  at  the  heart  of  every  joint  program  that  can  be  seen  in  the 
following  narrative,  which  summarizes  the  experiences  of  a  number  of  joint  and  joint-style 
programs  that  the  SEI  has  worked  with: 

A  joint  program  has  six  stakeholder  programs  all  planning  to  integrate  the 
joint  infrastructure  software  that  is  being  developed  to  meet  a  common 
baseline  set  of  requirements.  However,  each  stakeholder  program  then  also 
requests  that  one  or  more  significant  new  requirements  be  added  to  satisfy 
some  custom  needs  of  that  specific  stakeholder  program.  Although  reluctant, 
the  joint  program  manager  agrees  to  the  new  requirements  out  of  fear  of 
losing  stakeholder  programs,  who  might  leave  the  joint  program  to  build  their 
own  custom  software.  As  development  proceeds,  the  additional  requirements 
and  their  resulting  design  changes  and  incremental  development  significantly 
increase  the  total  cost,  schedule,  complexity,  and  risk  of  the  joint 
development  effort.  As  the  schedule  begins  to  slip,  one  stakeholder  program 
realizes  that  the  joint  program  has  put  the  stakeholder  in  danger  of  missing  its 
own  schedule,  and  so  it  leaves  the  joint  program  to  develop  its  own  software. 


2  The  original  story  of  the  “Tragedy  of  the  Commons”  from  the  19th  century  envisions  a  group  of 
herders  sharing  an  area  of  grazing  land  called  a  commons.  If  one  herder  decides  to  graze  an  extra 
animal,  then  that  herder  receives  more  benefit  from  the  commons  than  the  others,  and  at  no 
additional  cost  to  himself.  However,  if  all  of  the  herders  follow  suit  and  add  more  animals  according  to 
the  same  reasoning,  they  eventually  reach  the  point  where  the  grass  is  eaten  faster  than  it  can  grow, 
the  cattle  begin  to  starve,  and  ultimately  all  of  the  herders  lose  their  livelihood. 
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Although  one  stakeholder  program  has  left  the  joint  program,  the  incremental 
cost  of  the  more  complex  architecture  that  was  designed  to  support  the 
stakeholder’s  desired  capability  cannot  be  recouped.  The  schedule  delays 
from  the  increased  complexity  and  risk  impact  the  remaining  stakeholder 
programs  as  well,  and  soon  another  stakeholder  program  chooses  to  leave 
the  joint  program.  Exacerbated  by  the  effort  spent  in  re-planning  the  joint 
effort  each  time  a  stakeholder  program  leaves,  costs  continue  to  escalate, 
and  the  development  schedule  lengthens.  The  remaining  stakeholder 
programs  begin  to  reconsider  their  participation  in  the  joint  program,  and 
ultimately  participation  unravels  and  collapses. 

With  this  narrative  in  mind,  a  joint  program  can  be  viewed  as  a  “Tragedy  of  the 
Commons”  in  which  the  commons  is  the  development  resource  of  the  joint  program  office 
and  the  contractor.  The  entire  program  and  the  stakeholder  programs  are  collectively  worse 
off  if  the  stakeholder  programs  choose  to  exploit  the  development  resource  for  their 
individual  gain  by  insisting  on  having  custom  requirements  developed. 

It  is  important  to  note  that  a  “Tragedy  of  the  Commons”  situation  does  not  always 
occur  in  a  joint  program.  It  may  be  the  case  that  strong  leadership  from  the  joint  program 
manager,  or  a  highly  cooperative  culture  within  the  program,  will  prevent  it  from  happening. 
However,  given  the  fact  that  the  incentives  align  to  favor  unilateral  action  by  the  stakeholder 
programs  and  their  services,  unless  specific  preventative  steps  are  taken,  preventing  this 
social  trap  is  more  likely  to  be  the  exception  rather  than  the  rule. 

The  next  section  provides  context  for  the  creation  of  a  system  dynamics  model  of 
this  behavior. 

System  Dynamics  Background 

The  system  dynamics  method  helps  analysts  model  and  analyze  critical  behavior  as 
it  evolves  over  time  within  complex  socio-technical  domains.  A  key  tenet  of  this  method  is 
that  the  dynamic  complexity  of  critical  behavior  can  be  captured  by  the  underlying  feedback 
structure  of  that  behavior.  The  boundaries  of  a  system  dynamics  model  are  drawn  so  that  all 
of  the  enterprise  elements  necessary  to  generate  and  understand  problematic  behavior  are 
contained  within  them.  The  method  has  a  long  history,  as  described  in  Sterman  (2000)  and 
Meadows  (2008). 

System  dynamics  and  the  related  area  of  systems  thinking  encourage  the  inclusion 
of  “soft”  factors  in  the  model  such  as  policy,  procedural,  administrative,  and  cultural  aspects. 
The  exclusion  of  soft  factors  in  other  modeling  techniques  effectively  treats  their  influence  as 
negligible,  which  is  often  an  inappropriate  assumption.  This  holistic  modeling  perspective 
helps  identify  mitigations  to  problematic  behaviors  that  are  often  overlooked  by  other 
approaches. 

Figure  1  summarizes  the  notation  used  by  system  dynamics  modeling.  The  primary 
elements  are  variables  of  interest,  stocks  (which  represent  collection  points  of  resources), 
and  flows  (which  represent  the  transition  of  resources  between  stocks).  Signed  arrows 
represent  causal  relationships,  where  the  sign  indicates  how  the  variable  at  the  arrow’s 
source  influences  the  variable  at  the  arrow’s  target.  A  positive  (S)  influence  indicates  that 
the  values  of  the  variables  move  in  the  same  direction,  whereas  a  negative  (O)  influence 
indicates  that  they  move  in  opposite  directions.  A  connected  group  of  variables,  stocks,  and 
flows  can  create  a  path  that  is  referred  to  as  a  feedback  loop.  There  are  two  types  of 
feedback  loops:  balancing  and  reinforcing.  The  type  of  feedback  loop  is  determined  by 
counting  the  number  of  negative  influences  along  the  path  of  the  loop.  An  odd  number  of 
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negative  influences  indicates  a  balancing  loop;  an  even  (or  zero)  number  of  negative 
influences  indicates  a  reinforcing  loop. 

Significant  feedback  loops  identified  within  the  model  described  here  are  indicated  by 
a  loop  symbol  and  a  loop  name  in  italics.  Balancing  loops — indicated  with  the  label  B 
followed  by  an  identifying  number  in  the  loop  symbol — describe  aspects  of  the  system  that 
oppose  change,  seeking  to  drive  variables  to  some  equilibrium  goal  state.  Balancing  loops 
often  represent  actions  that  an  organization  takes  to  manage,  or  mitigate  a  problem. 
Reinforcing  loops — indicated  with  a  label  R  followed  by  a  number  in  the  loop  symbol — 
describe  system  aspects  that  tend  to  drive  variable  values  consistently  either  upward  or 
downward.  Reinforcing  loops  often  represent  the  escalation  of  problems  but  may  include 
problem  mitigation  behaviors. 


Vaiiable- anything  of  interest  in  the  problem  being 
modeled. 

Gliost  Variable  —  variable  acting  as  a  placeholder 
for  a  variable  occurring  somewhere  else 

Positive  Influence  —  values  of  variables  move  in 
the  same  direction  (e.g,,  source  increases,  target 
increases) 

Negative  Influence— values  of  variables  move  in 
the  opposite  direction  (e.g.,  source  increases,  the 
target  decreases) 

Delay -significant  delay  from  when  Nterl  changesto 
when  Var2  changes 

Balancing  Loop  -  a  feedback  loop  that  moves 
variable  values  to  a  goal  state;  loop  color  identifies 
circular  influence  path 

Reinforcing  Loop  -  a  feedback  loop  that  moves 
variable  values  consistently  upward  or  downward; 
loop  color  identifies  circular  influence  path 

Stock  -  special  variable  representing  a  pool  of 
materials,  money,  people,  or  other  resources. 

Flow  -  special  variable  representing  a 
process  that  directly  adds  to  or  subtracts  from 
a  stock. 

Cloud  -  source  or  sink  (represents  a  stock 
outside  the  model  boundary) 

Figure  1.  System  Dynamics  Notation 

The  next  section  discusses  how  the  system  dynamics  modeling  process  was  used  to 
elicit  a  detailed  understanding  of  joint  program  behavior  from  subject  matter  experts. 

Workshop  With  Domain  Experts 

A  series  of  problem  elaboration  workshops 3  is  being  used  as  the  primary  method  for 
gaining  feedback  from  acquisition  subject  matter  experts  on  the  current  system  dynamics 
model,  and  for  eliciting  suggestions  for  additional  potential  improvements.  To  date,  a 
shortened  pilot  version  of  the  problem  elaboration  workshop  has  been  conducted  with 
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3  These  workshops  are  covered  by  the  Carnegie  Mellon  University  (CMU)  human  subject  research 
policy,  and  protocol  HS1 2-237  for  conducting  these  workshops  has  been  approved  by  the  CMU 
Institutional  Review  Board  (IRB).  Nothing  discussed  at  the  workshops  is  tied  to  a  specific  individual  or 
organization. 
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internal  SEI  acquisition  experts  as  well  as  a  full  two-day  workshop  with  program  office  and 
contractor  personnel  from  a  single  joint  program. 

The  problem  elaboration  workshops  are  intended  to  consist  of  personnel  drawn  from 
a  single  joint  program.  Ideally  each  workshop  will  include  a  mix  of  program  management 
and  technical  personnel  as  well  as  personnel  from  both  the  acquirer  and  developer  side. 

The  workshops  last  approximately  two  days  in  order  to  cover  a  substantial  portion  of  the 
relevant  material.  The  top-level  causal  loop  diagram  of  the  dynamic  is  reviewed  as  a  high- 
level  abstraction  of  the  model  because  reviewing  the  entire  system  dynamics  model  is  not 
feasible  for  the  acquisition  subject  matter  experts. 

There  are  two  primary  goals  for  each  problem  elaboration  workshop:  (1)  discuss  the 
current  top-level  loops  in  the  model  causal  loop  diagram  and  have  the  participants  rate  the 
importance  and  accuracy  of  each  loop  using  a  Likert  scale  and  (2)  gain  insight  from  the 
participants  on  any  loops/interactions  that  may  have  been  overlooked.  The  initial  workshop 
focused  on  a  joint  program  designed  to  provide  a  joint  communication  capability  needed  by 
several  services  that  was  to  be  deployed  on  a  number  of  different  platforms  to  allow  for 
effective  communication  between  platforms  belonging  to  multiple  services.  The  participants 
included  personnel  who  had  worked  at  the  government  program  office  and  personnel  from 
the  prime  contractor.  The  workshops  were  effective  in  achieving  their  goals,  and  some  of  the 
results  are  summarized  as  follows. 

Goal  1:  Rating  the  top-level  loops.  After  presentation  and  discussion  of  all  of  the  top- 
level  loops  in  the  CLD  of  the  large  model  (see  Table  1  in  Appendix  A  for  high-level 
descriptions  of  those  loops  and  Figure  1 1  in  Appendix  B  for  a  graphical  depiction),  ratings 
were  obtained  from  all  participants.  Nine  of  the  12  loops  in  the  CLD  (75%)  were  rated  above 
moderately  important.  In  seven  (i.e.,  58%)  of  the  loops,  the  average  accuracy  score  was 
rated  above  moderately  accurate.  Of  these  seven  loops  rated  above  moderately  accurate, 
four  of  these  loops  (33%  of  the  original  12)  were  rated  above  very  accurate.  For  all  12  loops, 
at  least  one  of  the  four  participants  rated  themselves  as  extremely  experienced  in  this  area, 
and  all  loops  had  at  least  two  participants  who  rated  themselves  as  very  or  extremely 
experienced.  Based  on  the  feedback  from  the  participants,  one  section  of  the  CLD  that 
scored  lower  in  importance  was  modified  in  order  to  change  how  stakeholder  programs  may 
influence  others  to  defect,  or  leave  the  joint  program. 

Goal  2:  Overlooked  loops/interactions.  The  workshop  participants  discussed  nine 
additional  interactions  that  they  thought  had  been  important  on  their  joint  program.  The  top 
area  they  thought  should  be  added  addressed  launching  the  program  properly.  The  model 
was  modified  to  address  this  area,  and  additional  ways  of  implementing  this  concept  are 
being  explored.  A  second  area  that  was  identified  as  needing  to  be  addressed  is  the  level  of 
capability  of  the  government  staff,  and  this  has  been  added  to  the  model  as  well. 

Feedback  from  actual  program  personnel  is  critical  to  ensuring  that  the  model 
includes  the  most  important  top-level  interactions.  It  is  also  critical  to  tuning  the  model 
parameters  to  best  simulate  the  performance  of  joint  programs.  Additional  problem 
elaboration  workshops  are  planned  for  the  near  future  to  continue  to  refine  the  model. 

The  Joint  Program  Simulation  Model 

As  described  previously,  the  problem  elaboration  workshop  attendees  were 
presented  with  a  CLD  that  already  described  many  aspects  of  joint  program  behavior.  The 
feedback  from  these  domain  experts  made  it  possible  to  assess  the  most  important  aspects 
of  the  joint  program  problem,  many  of  which  were  included  in  the  original  CLD  and  some  of 
which  were  not.  This  information  was  used  to  develop  a  simpler  and  more  focused  CLD  that 
better  represents  the  inherent  social  dilemma  and  other  central  aspects  of  the  joint  program 
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dilemma  as  seen  by  the  workshop  participants.  Appendix  B  contains  this  refined  CLD.4  As 
additional  workshops  are  conducted,  other  aspects  may  be  included  or  excluded  from  the 
CLD  based  on  the  findings  of  those  workshops. 

The  only  loops  retained  in  this  simpler  model  are  the  stakeholder  custom 
requirements  acceptance  (B3),  pressure-induced  rework  (R3),  and  pressure-induced 
attrition  (R4),  as  described  in  Appendices  A  and  B.  The  first  two  of  these  were  the  top  two 
rated  feedback  loops  at  the  workshop.  The  third,  which  is  closely  related  to  the  second, 
occurs  in  most  joint  programs  and  causes  significant  turmoil  and  lost  productivity.  Also 
included  is  one  of  the  two  highest  rated  extensions  proposed  to  the  original  model:  The 
inclusion  of  Joint  Program  Office  (JPO)  efforts  to  keep  the  joint  program  sold  to  stakeholders 
was  deemed  a  key  contributing  factor  to  endemic  problems  and  inefficiencies.  The  top- 
rated  extension  that  was  suggested  at  the  workshop,  the  distinction  between  acquiring 
capabilities  as  opposed  to  acquiring  systems,  will  be  addressed  explicitly  in  future  versions 
of  the  model. 

The  system  dynamics  method  provides  a  way  of  implementing  a  CLD,  so  as  to 
further  explore  the  implications  of  the  causal  structure  as  it  is  elaborated  in  more  detail. 
These  implications  are  assessed  through  simulation  (execution)  of  the  model.  In  addition  to 
the  confidence  gained  in  the  CLD  during  the  workshops,  simulation  can  result  in  additional 
confidence  that  the  causal  structure  can  indeed  produce  the  behavior  implied  by  the 
qualitative  CLD.  Once  the  model  has  been  shown  to  exhibit  the  expected  behavior, 
workshop  interactions  can  help  ensure  that  it  does  so  for  the  correct  reasons.  This  level  of 
validation  then  allows  the  analyst  to  use  the  model  to  test  alternate  solutions  to  the  problem 
using  the  system  dynamics  simulation  capability. 

The  simulation  and  analysis  of  the  joint  program  model  is  still  ongoing,  and  it  is  the 
initial  results  of  that  effort  that  are  presented  here.  The  feedback  that  was  received  in  the 
initial  problem  elaboration  workshop  made  it  possible  to  simplify  and  focus  the  original 
simulation  model  that  had  been  developed.  The  three  primary  segments  of  the  current 
simulation  model  are  described  in  order:  the  Stakeholder  Program  Segment,  the  Joint 
Program  Office  (JPO)  Segment,  and  the  Developer  Segment.  Each  of  the  stakeholder 
programs,  the  JPO,  and  the  developer  have  reasons  to  be  at  least  comparatively  satisfied 
based  on  the  progression  of  events  thus  far,  early  on  in  the  joint  program  acquisition. 
However,  as  will  be  seen  in  the  subsequent  section,  Systemic  Effects,  their  relative 
satisfaction  will  be  spoiled  due  to  the  diminishing  returns  associated  with  joint  program 
expenditures. 

•  The  current  model  makes  the  following  assumptions  about  the  joint 
acquisition  program: 

•  The  timeline  of  the  simulation  is  120  months — 10  years — but  the  conclusion 
of  the  project  may  be  significantly  short  of  that,  and  vary  depending  on  the 
input  parameters.  Milestone  B  occurs  12  months  into  the  simulation,  and  that 
is  when  the  development  contract  is  awarded. 

•  The  joint  program  has  three  stakeholder  programs  that  negotiate  with  the 
JPO  for  their  own  custom  requirements  separate  from  a  set  of  baseline 
requirements.  The  stakeholder  programs  are  referred  to  abstractly  as  SI,  S2, 
and  S3. 


4  Note  that  CLDs  and  system  dynamics  models  share  a  similar  notation.  The  primary  difference  is  that 
CLDs  do  not  include  stocks  or  flows.  They  are  strictly  qualitative  and  so  are  not  executable. 
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•  Funding  for  the  joint  program  is  spent  strictly  on  development  activities.  JPO 
staff  can  rotate  out  and  be  hired  in,  but  the  staff  levels  stay  at  generally  the 
same  level  and  do  not  consume  funding  (e.g.,  they  are  on  overhead,  as  far  as 
the  model  is  concerned). 

•  Developer  staff  are  separated  into  new  staff  versus  experienced  staff,  each 
with  their  own  levels  of  productivity  (i.e.,  computer  software  configuration 
items  (CSCIs)5  developed/tested  per  month)  and  monthly  costs.  Experienced 
staff  may  have  their  time  partially  consumed  by  training  new  staff. 

These  assumptions  may  be  relaxed  in  future  revisions  to  the  model  to  allow  a  broader  range 
of  behaviors  to  be  tested. 

It  should  be  noted  that  although  the  model  described  as  follows  has  been  refined 
both  by  the  problem  elaboration  workshop  sessions  and  through  the  acquisition  experience 
of  the  modeling  team  itself,  this  model  has  not  yet  been  validated  with  historical  joint 
program  data  to  help  quantify  the  relationships  between  the  model  variables.  This  validation 
will  be  conducted,  but  at  this  point,  the  model  should  be  viewed  as  providing  only  tentative 
support  for  the  causal  hypothesis. 

Stakeholder  Program  Segment 

A  primary  concern  of  the  stakeholder  programs  is  getting  their  (custom)  requirements 
implemented  by  the  joint  program  so  that  they  have  the  most  usable  system  possible  when 
the  joint  program  completes  development.  There  is  a  fair  amount  of  negotiation  going  on 
during  these  times  between  the  joint  program  office  and  the  stakeholder  programs,  and  the 
initial  model  is  based  on  the  foundations  of  negotiation  and  cooperation  theory.  Other  work 
in  developing  system  dynamics  models  has  leveraged  some  of  this  theory  in  the  past.  This 
model  is  based  explicitly  on  models  developed  by  Darling  and  Richardson  (1990). 

As  illustrated  in  Figure  2,  stakeholder  program  decision-making  is  based  on  the 
following: 

•  Stakeholder  program  gain  (the  inner  loop  in  the  figure).  The  extent  to  which 
the  stakeholder  program’s  custom  requirements  are  implemented  in  the  joint 
system.  In  terms  outlined  by  Darling,  this  gain  limits  the  stakeholder 
program’s  problem  potential.  An  effect  function6  is  used  to  capture  the 
framing  effects  of  Darling’s  model,  which  is  used  to  determine  whether  the 
extent  of  the  stakeholder  program’s  gain  is  viewed  positively  or  negatively. 

•  Stakeholder  program’s  relative  gain  (the  outer  loop  in  the  figure).  The 
stakeholder  program’s  satisfaction  is  also  dependent  on  how  much  they 
perceive  others  are  gaining  relative  to  their  own  gain.  If  they  think  others  are 
getting  proportionally  more,  then  they  will  be  less  satisfied  even  if  they  are 
still  getting  their  own  needs  met  adequately.  This  is  a  refinement  of  Darling’s 
model,  which  was  based  on  a  weighted  sum  of  the  gain  for  self  and  the 
perceived  gain  of  other  stakeholder  programs. 

o  A  more  recent  perception  of  gains  weighs  more  in  stakeholder  program 
decision-making  than  older  perceptions.  This  relates  to  the  moving 
average  used  in  the  Darling  model,  which  models  how  past  outcomes 
influence  present  expectations. 


5  A  CSCI  is  a  collection  of  software  that  supports  a  specific  function  for  the  end  user. 

6  An  effect  function  is  a  device  used  in  system  dynamics  modeling  that  explicitly  describes  the 
mathematical  relationship  between  two  specific  model  variables  over  time. 
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o  The  possibility  that  a  stakeholder  program  may  have  only  a  limited 
understanding  of  other  stakeholder  programs’  gains  (Darling’s  “Fixed 
Pie  Bias”)  is  handled  with  a  weighted  formula.  To  the  extent  that 
understanding  is  incomplete  (i.e.,  knowledge  of  other’s  gain  is  less  than 
1 ),  a  stakeholder  program  assumes  that  their  loss  is  the  other 
stakeholder  program’s  gain. 

Initial  discussion  with  joint  program  decision-makers  suggests  that  concern  for 
fairness,  as  described  in  Darling’s  model,  is  not  a  primary  factor  in  stakeholder  program 
decision-making,  so  it  has  been  omitted  from  the  simplified  model  presented  here.  It  is, 
however,  still  a  factor  in  the  larger  model  being  developed. 

A  stakeholder  program’s  satisfaction  influences  both  the  extent  of  their  buy-in  to  the 
joint  program  and  their  cooperation  with  the  joint  program  goals.  Both  buy-in  and 
cooperation  with  the  joint  program  are  needed  to  keep  the  program  viable.  When  either  is 
lagging,  the  JPO  will  tend  to  implement  more  of  the  stakeholder  program’s  custom 
requirements  to  keep  the  stakeholder  program  engaged. 


Figure  2.  Stakeholder  Programs  Negotiate  for  Custom  Requirements  Beyond  Baseline 

This  effect  can  result  in  an  escalation  of  custom  requirements,  which  of  course  must 
then  be  integrated  with  the  original  requirements.  The  model  initial  settings  are  set  to  an 
equilibrium.  At  Month  1 8,  to  test  the  behavior  of  the  model,  the  demands  of  stakeholder  SI 
are  stepped  up  to  a  level  of  0.8  on  a  scale  of  0  to  1 .  This  perturbation  from  equilibrium 
shows  in  Figure  3  that  increases  in  one  stakeholder  program’s  demands  leads  to  increases 
in  other  stakeholder  programs’  demands.  Although  the  levels  do  not  rise  to  the  same 
degree,  the  escalation  of  custom  requirements  that  result  are  necessary  from  the  joint 
program  perspective  in  order  to  maintain  stakeholder  programs’  buy-in  and  prevent 
stakeholders  from  defecting. 
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Fraction  of  Custom  Requirements  that  are  Accepted  by  the  JPO 


fraciioncratoairaqiJtcipfidJSl]  :  Cusrefi  - 1 - 1 - 1 - * - * 

5r*ciioGC^tofflir«qitcctp*d(S23 :  Owtct  - 2 - 2 - 2 - 2 - 

5tk  lion  r*q»  ice  ;  Qztttt  -3 - 3 - 3 - 3 - 3 — 


Figure  3.  Increase  in  Custom  Requirements  Acceptance  for  SI  With  Subsequent  Rise  for 

S2  and  S37 

In  the  Darling  model,  this  behavior  reflects  the  “competitive  drift”  possible,  where  one 
negotiator  is  pitted  directly  against  another  and  the  interaction  between  negotiators 
becomes  increasingly  acrimonious.  In  the  joint  program  case,  the  JPO  may  feel  compelled 
to  give  in  to  stakeholder  program  demands  across  the  board,  directly  supporting  the  creation 
and  reinforcement  of  the  underlying  social  dilemma.  With  greater  support  being  given  to 
their  individual  needs,  the  stakeholder  programs  remain  relatively  satisfied. 

Joint  Program  Office  (JPO)  Segment 

The  benefit  of  keeping  stakeholder  programs  “bought  in”  to  the  joint  program  is 
evident  in  Figure  4.  More  engaged  program  stakeholders  promote  DoD  buy-in.  Once  the 
development  starts,  especially  with  the  additional  custom  requirements  accepted,  plus-ups 
on  funding  and  extensions  to  the  schedule  are  usually  necessary  to  implement  the 
additional  functionality. 


7  This  and  subsequent  graphs  were  generated  using  the  Vensim  modeling  tool.  These  are  all 
behavior-over-time  graphs,  and  as  such,  the  x-axis  for  these  graphs  is  specified  in  months  (120 
months — 10  years — is  the  duration  of  this  simulation).  Each  simulation  run  is  specified  as  individual 
graphs  distinguished  with  a  number  label  (1  through  3  in  Figure  3),  as  specified  in  the  legend  below 
the  graph. 
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Figure  4.  JPS  Benefits  From  Increased  Stakeholder  Program  Buy-In  by  Keeping  the 

Program  Alive 


T otal  Funding 


0  12  24  36  48  60  72  84  96  108  120 

Time  (\bnth) 


Total  Finding :  Curie  ra  H - =1 - 1 - 1 - =1 - 1 - 4 - 4 

Total  Finding :  Baseline  —2 - 2 - 2 - 2 - 2 - 2 - 2 - 

Figure  5.  Additional  Funding  Increments  to  Implement  Expanded  Scope 
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.Announced  Completion  Date 


Amoinetd  Compieticm  Date  :  Ctrrent  -1 - 1 - I - 1 - t - F 

Announced  Completion  Date  :  Baseline  — 2 - 2 - 2 - 2 - 2 - 


Figure  6.  Additional  Schedule  Extensions  to  Implement  Expanded  Scope 


Developer  Segment 

The  additional  development  work  generated  due  to  the  additional  custom 
requirements  from  the  stakeholder  platforms  is  shown  in  the  middle  of  Figure  7.  This 
additional  development  work,  along  with  the  development  work  from  the  originally  planned 
baseline,  is  added  to  the  development  work  remaining.  Both  development  and  testing  work 
is  accomplished  based  on  the  productivity  of  the  development  staff,  shown  on  the  left  side  of 
the  figure. 


Figure  7.  Development  Staff  Managed  to  Complete  Development  Work 


Development  staff  is  split  between  new  hires  and  experienced  staff,  with  some 
training  period  (possibly  on-the-job)  needed  to  transition  from  new  to  experienced.  The 
development  productivity  levels  of  new  and  experienced  staff  differs,  with  experienced  staff 
spending  some  of  their  time  training  the  newer  staff.  All  charges  made  by  the  staff  for  their 
time  working  on  the  project  are  reflected  in  the  cumulative  contractor  (i.e.,  developer) 
revenue.  As  shown  in  Figure  8,  the  contractor’s  revenue  rises  well  above  the  baseline 
levels,  partially  due  to  implementing  the  additional  custom  requirements  demanded  by  the 
stakeholder  programs.  In  this  context,  assuming  that  the  contractual  negotiations  are 
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providing  additional  revenue  for  the  additional  employees,  the  contractor  is  willing  (if  not 
even  happy)  to  employ  more  staff  for  a  longer  period. 


Cumulative  Developer  Revenue 


Cvoulativ*  Develop*?  Re  vena*  :  Current  - 5 - t - 1 - 1 — 

CiOTiilative  Dev*Iop«  Re  van  tie  :  Baseline  - 2 - 2 - 2 - 2 


Figure  8.  Developer  Revenue  Rises  Well  Above  the  Baseline  Level8 
Systemic  Effects 

Although  the  stakeholder  programs,  the  JPO,  and  the  developer  accomplish 
important  objectives  in  their  own  domains,  these  objectives  act  as  a  trap  for  joint  program 
decision-makers  that  can  potentially  lead  to  the  demise  of  the  joint  program.  Figure  9  shows 
the  diminishing  returns  related  to  the  joint  program  investment  to  develop  the  extended  joint 
system.  As  the  number  of  custom  requirements  accepted  for  each  stakeholder  program 
increases  along  the  x-axis,  the  average  cost  per  CSCI  increases  by  a  factor  of  5  to  10  over 
the  average  cost  per  CSCI  in  the  baseline  development.  Another  dimension,  along  the  z- 
axis,  shows  that  as  the  realism  of  schedule  setting  decreases  from  1  to  0.1 ,  the  CSCI  cost 
ratio  declines  even  further.  As  a  result  of  the  simulation  and  analysis  of  this  scenario, 
representations  of  complex  decision  surfaces  such  as  shown  in  Figure  9  allow  decision¬ 
makers  to  understand  the  interactions  between  multiple  factors  within  a  system,  and  to 
understand  the  range  of  possible  outcomes  based  on  various  actions. 


8  Development  is  complete  about  Month  30  in  the  Baseline  simulation  run  and  about  Month  47  in  the 
Current  run. 
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Figure  9.  Systemic  Result:  Diminishing  Returns  in  Development  Effort  Lead  to  Cost 

Increases  for  Program 


The  overview  model,  shown  in  Figure  10,  integrates  the  stakeholder  program 
segment,  the  JPO  segment,  and  the  developer  segment  described  previously.  The  model 
also  illustrates  the  primary  influences  causing  the  diminishing  returns: 


•  Complexity-Induced  Rework  (in  blue  in  the  lower  middle  of  the  figure) — The 
system  complexity  that  results  from  program  stakeholder  custom 
requirements  decreases  average  development  productivity  and  increases  the 
rates  of  defect  injection  during  development.  The  increased  system 
complexity  increases  the  complexity  of  developing  individual  CSCI  for  a 
variety  of  reasons,  making  development  take  longer  and  be  more  error  prone. 

•  JPO  Staffing  Effects  on  Program  Execution  (in  green  in  the  lower  middle  of 
the  figure) — The  resource  demands  on  the  JPO  staff,  as  described  previously 
in  the  JPO  segment,  causes  two  primary  problems  for  the  developers.  First, 
the  JPO  staff  is  not  as  responsive  to  developer  demands  for  guidance,  and 
for  review  and  feedback  on  development  artifacts.  This  reduces  the  average 
developer  productivity.  The  second  effect  is  that  the  JPO  staff  shortcuts  the 
quality  of  their  guidance  and  review  process.  This  leads  to  lower  quality  in  the 
development,  and  greater  amounts  of  rework. 


•  Pressure-Induced  Rework  (the  red  reinforcing  feedback  loop) — The 

expansion  of  the  joint  system  scope  leads  to  the  need  for  extensions  to  the 
schedule  well  beyond  those  planned  for  the  original  baseline  system. 
Although  the  need  for  schedule  extensions  is  widely  recognized,  they  may 
come  infrequently  at  unpredictable  times,  and  only  if  decision-makers  remain 
adequately  bought  in.  The  result  is  intense  schedule  pressure,  which  may  be 
evident  even  early  in  the  program  if  the  initial  schedule  was  unrealistic.  Such 
schedule  pressure  can  lead  to  bypassing  some  quality  processes,  and  to  the 
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generation  of  higher  levels  of  rework.  This  acts  in  a  reinforcing  manner  as 
schedule  pressure  escalates  even  further. 

•  Pressure-Induced  Attrition  (the  purple  reinforcing  feedback  loop) — 
Development  staff  may  suffer  the  most  from  schedule  pressure.  When 
development  staff  are  in  high  demand,  attrition  may  grow.  Despite  new 
development  staff  being  hired,  the  average  and  thus  the  overall  productivity 
may  fall,  making  it  even  harder  to  meet  schedule  demands.  This  reinforcing 
dynamic  exacerbates  the  problem  further. 

This  section  described  the  hypotheses  about  why  joint  programs  can  get  trapped  into 
a  development  of  diminishing  returns.  The  four  causes  for  these  diminishing  returns, 
described  previously,  provide  a  view  of  what  can  go  wrong.  Mitigation  of  this  problem  may 
involve  developing  a  means  to  avoid  falling  into  the  trap  in  the  first  place  or  for  reducing  the 
negative  consequences  associated  with  falling  in  the  trap.  The  next  section  describes  some 
of  the  considerations  regarding  problem  mitigation. 

Mitigations  for  the  Joint  Program  Dilemma 

The  rationale  for  identifying  a  possible  inherent  social  dilemma  at  work  within  the 
structure  of  a  joint  program  is  to  understand  the  mechanism  by  which  these  types  of 
acquisition  programs  can  encounter  difficulties.  Once  the  mechanism  has  been  confirmed, 
there  is  a  large  set  of  mitigations  and  solution  approaches  that  have  been  developed  in 
different  academic  disciplines  such  as  game  theory,  behavioral  economics,  social  science, 
and  social  psychology,  with  each  addressing  differences  in  the  specifics  of  the  instance  of 
the  dilemma.  Elinor  Ostrom  received  the  2009  Nobel  Prize  in  Economics  for  her  study  of 
innovative  solutions  that  have  evolved  in  different  cultures  to  address  differing  instances  of 
the  “Tragedy  of  the  Commons.”  However,  these  academic  solutions  are  not  well  known  to 
the  software-intensive  acquisition  or  software  development  communities  and  thus  have  not 
yet  been  studied  in  the  context  of  acquisition  programs,  so  their  applicability  is  still  unknown. 
The  goal,  however,  remains  the  same — to  deploy  higher  quality  systems  to  the  field  in  a 
more  timely  and  cost-effective  manner. 

The  research  literature  organizes  the  solutions  to  social  dilemmas  such  as  the 
“Tragedy  of  the  Commons”  into  three  classes: 

•  Motivational.  Motivational  solutions  assume  that  participants  are  not  exclusively 
self-interested  and  thus  care  about  the  consequences  of  their  actions  on  other 
participants.  Because  of  this,  such  concerns  as  values  and  group  identity,  as  well 
as  communication,  can  be  effective. 

•  Strategic.  Strategic  solutions  assume  that  participants  are  exclusively  self- 
interested  and  so  require  that  the  participants  influence  how  the  other 
participants  behave,  thus  producing  a  better  outcome  for  themselves.  Robert 
Axelrod  (1984)  provided  three  ingredients  for  such  approaches:  (1)  long-term 
relationships  among  the  participants  (so  that  all  expect  shared  dilemmas  in  their 
future),  (2)  that  the  participants  can  identify  one  another,  and  (3)  that  participants 
are  aware  of  the  past  behavior  of  each  other. 

•  Structural.  Structural  solutions  require  changing  the  rules  of  the  situation  so  that 
the  nature  of  the  dilemma  also  changes.  The  most  significant  difficulties  with 
applying  structural  solutions  is  that  (1)  they  require  a  level  of  authority  to 
implement,  (2)  they  may  bring  about  resistance  from  those  who  are  affected,  and 
(3)  they  require  methods  (with  accompanying  costs)  to  ensure  compliance  with 
the  new  rules. 
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The  first  two  classes  (i.e.,  motivational  and  strategic)  do  not  require  changing  the 
fundamental  structure  of  the  situation,  and  as  a  result,  they  tend  to  be  simpler  to 
implement — although  their  effectiveness  is  less  certain  when  compared  to  a  structural 
solution. 

To  discuss  one  common  approach  to  resolving  a  social  trap,9  the  use  of  an  authority 
to  manage  the  commons  is  widely  used  in  practice.  However,  this  approach  may  have  side 
effects,  depending  on  how  the  leader  was  selected  and  from  which  organization,  since  the 
perceived  objectivity  and  neutrality  of  the  leader  is  essential  to  their  acceptance  by  the 
participants. 

Another  widely  used  approach  is  privatization,  which,  like  the  use  of  authority,  also 
has  side  effects.  By  removing  the  social  aspect  of  the  social  dilemma,  it  eliminates  the 
interdependence  between  people  by  converting  shared  ownership  to  private  ownership. 
However,  this  would  result  in  each  of  the  stakeholder  programs  building  their  own  custom 
system,  which  is  antithetical  to  the  originally  intended  outcome. 

Another  approach  that  could  produce  a  better  outcome  might  be  altruistic 
punishment  (Fehr  &  Gachter,  2002).  In  altruistic  punishment,  cooperating  participants  may 
penalize  uncooperative  participants  through  some  mechanism  (such  as  withholding  a  small 
funding  increment)  at  a  small  cost  to  themselves.  Participants  seem  willing  to  do  this, 
despite  the  cost — and  even  if  it  will  yield  no  direct  material  gain  to  them.  Fehr  and  Gachter’s 
research  indicated  that  cooperation  increases  if  altruistic  punishment  is  possible  and  may 
break  down  if  it  is  ruled  out.  In  addition,  imposing  a  cost  on  the  administering  party 
disincentivizes  overuse,  making  it  self-correcting. 

Such  a  solution  could  help  to  avoid  the  requests  for  additional  capabilities  and 
prevent  the  downward  spiral  due  to  a  lengthening  schedule  and  increasing  cost,  complexity, 
and  risk,  thus  incentivizing  stakeholder  programs  to  stay  with  the  joint  program,  rather  than 
defect.  However,  this  particular  solution  to  the  social  trap  may  or  may  not  be  feasible  for  use 
on  a  joint  program. 

Another  way  of  addressing  a  social  trap  would  be  a  strategic  approach:  making  a 
series  of  small  changes  to  the  incentive  and  reward  structure  of  the  program,  such  as 
improving  communications,  making  negative  behaviors  more  visible  to  all  participants,  and 
similar  modifications.  Although  no  single  such  change  would  be  likely  to  significantly  mitigate 
the  problem,  it  may  be  that  the  aggregate  effect  of  many  small  changes  to  the  program 
structure,  when  taken  together,  could  have  a  substantial  positive  impact. 

Other  solutions  to  addressing  social  dilemmas  exist,  such  as  building  trust,  exclusion 
mechanisms,  assurance  contracts,  and  many  others.  The  choice  of  the  best  solution  will 
depend  on  the  specific  circumstances  surrounding  the  specific  joint  program  dilemma. 

The  defense  acquisition  system  itself  poses  some  significant  challenges  to 
successfully  mitigating  the  types  of  problems  that  are  inherent  to  joint  programs  and 
common  infrastructure  programs.  When  looking  at  the  structural,  strategic,  and  motivational 
classes  of  solutions  to  social  dilemmas,  it  is  apparent  that  motivational  solutions,  while 
attractive  due  to  their  generally  lower  level  of  effort  to  implement,  may  have  little  ability  to 
effect  change  if  the  participants  have  substantial  self-interest.  The  knowledge  that  “the 
complicated  acquisition  system  generates  staggering  bargaining  and  coordination  costs” 
such  as  “bureaucratic  politics  including  inter-service  rivalry,  Joint  service  logrolling”  (Lindsay, 
2006)  make  a  belief  in  the  services  having  low  levels  of  self-interest  seem  unlikely.  Strategic 
solutions  are  more  pragmatic  but  rely  largely  on  the  reputation  of  individuals  and  longer  term 


9  Social  traps  were  discussed  in  the  section  Misaligned  Incentives  in  Acquisition. 
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relationships  between  negotiating  parties,  both  of  which  are  in  short  supply  in  a  military 
where  “the  average  tenure  for  program  management  in  DoD  is  only  18  months”  (McConnell, 
Sickler,  &  Yang,  2004).  Structural  solutions  thus  may  appear  to  have  the  most  promise  of 
the  three  classes,  although  convincing  all  of  the  authorities  required  both  to  implement  and 
enforce  new  rules  on  all  parties  in  a  joint  program  context  may  prove  to  be  problematic. 

The  research  with  the  system  dynamics  model  of  joint  programs  that  is  being 
developed  involves  the  selection  of  some  of  the  most  promising  mitigation  and  solution 
approaches,  and  modeling  those  approaches  in  the  context  of  the  joint  program  model.  By 
assessing  the  ability  of  these  solution  approaches  to  mitigate  the  key  adverse  dynamics  that 
are  often  present  in  joint  programs,  it  will  be  possible  to  identify  a  set  of  the  most  promising 
approaches  that  could  be  applied  in  practice  to  try  to  avoid  these  issues  in  an  actual  joint 
acquisition  program. 
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Future  Work 

Some  of  the  possible  areas  for  future  work  will  involve  additional  refinement  and 
validation  of  the  simulation  model  through  review  and  feedback  by  joint  acquisition  domain 
experts,  as  well  as  calibration  with  historical  program  performance  data.  Once  sufficient 
confidence  in  the  model  is  gained  through  validation,  it  can  be  studied  further  to  understand 
how  the  different  key  model  variables  are  interrelated,  and  contribute  to  the  causes  of 
problematic  behaviors.  Complex  surfaces,  such  as  the  one  shown  in  Figure  9,  can  be 
created  to  give  a  sophisticated  understanding  to  decision-makers  as  to  how  multiple 
variables  interrelate  and  interact.  The  model  can  thus  be  used  as  a  management  decision 
aid  to  gain  an  understanding  of  what  might  occur  in  the  future  if  current  conditions  continue 
unchanged  and  to  explore  hypothetical  “what  if”  scenarios  based  on  potential  decisions  and 
events.  As  the  work  proceeds,  candidate  motivational,  strategic,  and  structural  mitigations  to 
the  problematic  dynamics  of  the  joint  program  social  dilemma  will  be  developed  and 
simulated  to  assess  their  effectiveness  and  viability,  and  to  help  develop  potential  new 
approaches  and  even  policy  recommendations  to  help  improve  the  execution  of  these  types 
of  programs. 

Although  no  model  can  accurately  predict  with  consistent  accuracy  the  future  states 
of  a  complex  dynamic  system  such  as  an  acquisition  program,  Donella  Meadows  (1974) 
pointed  out  that  “this  level  of  knowledge  is  less  satisfactory  than  a  perfect,  precise  prediction 
would  be,  but  it  is  still  a  significant  advance  over  the  level  of  understanding  permitted  by 
current  mental  models.” 

Conclusion 

This  paper  describes  the  results  of  a  preliminary  investigation  into  the  problems 
encountered  by  joint  acquisition  programs.  Through  interaction  with  joint  acquisition  experts, 
decision-makers,  and  stakeholders,  a  CLD  now  exists  that  represents  a  refined 
understanding  of  the  problem.  The  CLD  embodies  a  growing  comprehension  of  what 
happens  in  joint  acquisition  programs  and  why  the  stakeholder  programs,  the  JPO,  and  the 
developer  can  become  trapped  in  behaviors  that  make  rational  sense  to  the  participants  at 
the  time  but  can  lead  to  diminishing  returns  and  potentially  failure  for  the  program.  It 
describes  the  inherent  social  dilemma  that  exists  within  joint  programs — and  provides  the 
basis  for  a  better  understanding  of  the  problem  and  for  developing  ways  of  mitigating  it  to 
minimize  future  joint  program  challenges. 
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Appendix  A:  Feedback  Loops  Discussed  in  Workshop 

Table  1.  Loops  of  the  Original  CLD  Discussed  at  the  Problem  Elaboration 

Workshop 


Loop 

Name 

Description 

R1 

Stakeholder 

Bandwagon 

Low  stakeholder  satisfaction  can  lead  to  a  desire  to  defect,  as  well  as  attempts  to 
influence  other  stakeholders  to  defect,  causing  a  vicious  cycle  that  can  collapse  the 
joint  program. 

B1 

Membership 

Management 

Lack  of  stakeholder  support  can  lead  to  low  service  support,  especially  if  the  program’s 
value  to  the  service  is  low.  This  may  require  a  greater  “marketing”  effort  by  the  JPO  to 
sustain  stakeholder  support. 

B2 

Program 

Support 

Lowered  service  support  can  undermine  DoD  support,  requiring  still  more  JPO 
“marketing”  effort  to  keep  the  stakeholders  engaged. 

R2 

Stakeholder 
Confidence  in 
JPO 

Stakeholder  support  can  grow  as  the  progress  of  the  grogram  adheres  to  the  schedule 
set  forth.  However,  if  the  program  falls  behind  schedule,  stakeholders  may  become 
dissatisfied,  start  to  lose  confidence,  and  ultimately  even  defect. 

B3 

Stakeholder 

Custom 

Requirements 

Acceptance 

Stakeholders  are  especially  concerned  with  meeting  their  own  custom  requirements.  To 
the  extent  those  requirements  are  not  addressed,  the  stakeholders  may  insist,  and  the 
JPO  may  eventually  need  to  accept  their  requirements. 

B3b 

Stakeholder 

As  more  of  a  stakeholder’s  custom  requirements  are  accepted,  fairness  to  others  may 

ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-119- 


Concern  for 
Fairness 

come  into  play.  The  stakeholder  may  become  more  cooperative,  lowering  his/her 
demands  for  more  custom  requirements. 

B4 

Honey  Rather 
than  Vinegar 

The  JPO  may  resist  accepting  custom  requirements  if  the  stakeholder  becomes  too 
demanding.  The  stakeholder  may  then  reassess,  becoming  more  cooperative  if  he 
thinks  more  of  his  custom  requirements  will  be  accepted. 

R3 

Pressure- 

Induced 

Rework 

Accepting  custom  requirements  leads  to  expanded  program  scope.  Without  schedule 
relief  or  additional  staff,  this  puts  additional  pressure  on  workers,  potentially  causing 
them  to  bypass  quality  processes,  thus  resulting  in  more  rework. 

B5 

De-scoping 

To  reduce  schedule  pressure  and  try  to  get  development  back  on  track,  the  JPO  may 
eliminate  requirements  or  defer  them  to  a  later  development  phase. 

R4 

Pressure- 

Induced 

Attrition 

If  sustained,  excessive  schedule  pressure  can  disgruntle  developers,  leading  to 
attrition,  and  making  it  even  harder  to  meet  schedule  demands. 

R5 

Stakeholder 
Missing  their 
Schedule 

Delaying  the  schedule  past  the  stakeholder’s  need  date  for  the  capability  increases 
dissatisfaction,  and  can  be  a  primary  cause  of  defection. 

R6 

Stakeholder 

Escalating 

Costs 

Expanding  project  scope  can  lead  to  greater  shared  costs  to  each  stakeholder.  This 
may  increase  discontent  and  lead  to  greater  demands  to  meet  custom  stakeholder 
requirements,  especially  early  on. 

Appendix  B:  Simplified  Causal  Loop  Diagram 


Figure  1 1 .  Causal  Loop  Diagram  of  the  Joint  Program 
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